DataWarp: Building Applications Which Make Progress in an Inconsistent World

Some of the data available to today’s enterprise applications is inconsistent. This is an inevitable consequence of data being distributed and partially replicated in many locations. The usual approach to dealing with these imperfections is to attempt to drive them out at the application level, if not throughout the entire system. However, since at least some of the systems holding information are themselves unreliable, out of date or periodically disconnected, this is often futile. Nevertheless, considerable effort continues to be devoted to maintaining global data consistency. This paper describes an alternative approach to this problem for distributed systems which permits applications to operate in environments containing inconsistent data. Instead of taking the data available and attempting to extract a single, unique, correct view of the world, faced with inconsistencies, a DataWarp application constructs the collection of interpretations which can arise from its data. The application then adopts one of these views, together with the assumptions which accompany it and continues work on that basis. Since it acts on assumptions, the DataWarp application cannot guarantee that it never acts incorrectly. Instead it considers its recent work to be provisional in the expectation that eventually most of its actions will become definitive. However, on occasion, usually after receiving new data, the application may decide to change its assumptions and adopt an alternative view. When this happens, it may need to backtrack to an earlier state, voiding some provisional actions as it does so. It can then resume work based on its revised outlook. We describe the DataWarp architecture, discuss its implementation and describe an experiment in which a DataWarp application in an environment containing inconsistent data achieves considerably better results than its conventional counterpart.